A Vignette for the Transportation Safety Review Paper

In this review paper, we attempt to provide a comprehensive review on transportation research and optimization models. This vignette serves as the supplementary materials to create reproducible examples in the manuscript Bridging the Gap between Transportation Safety Research and its Incorporation inOptimization Models: a Detailed Review and Perspective. Submitted to Transportation Research Part C: Emerging Technologies.

This vignette includes examples on the following five aspects:

  1. Bibliographic analysis: Bibliometric Summary for Transportation Safety, Bibliographic Network Matrices - Journal Names, keyword co-occurrences network, a Conceptual Structure Map
  2. Extracting online transportation safety data: Crash-related data, traffic flow data, and weather data.
  3. Descriptive analytic tools: data visualization and clustering
  4. An example of clustering
  5. Statistical methods: logistic regression and Poisson regression

To maximize the readability of this vignette, we hided all R codes by default, but readers can look into any code by clicking the code button.

1 Bibliographic analysis

To perform a quick bibliometric analysis on Clarivate Analytics Web of Science (WoS), we searched transportation safety publications in WoS Core Collection using the following combination of words without limiting the document type, years, language:

("hazmat transportation" OR "real-time crash prediction" OR ("vehicle routing" AND safety))

This resulted in a downloadable plain .txt file (we named it ‘savedrecs.text’ under data folder), containing the full records with cited references for 992 results (conducted at 7/30/2018 - 11:08 am ET). We then used the R package bibliometrix to conduct a bibliometric analysis on this topic.

1.1 Bibliometric Summary for Transportation Safety

## 
## Converting your isi collection into a bibliographic dataframe
## 
## Articles extracted   100 
## Articles extracted   200 
## Articles extracted   300 
## Articles extracted   400 
## Articles extracted   500 
## Articles extracted   600 
## Articles extracted   700 
## Articles extracted   800 
## Articles extracted   856 
## Done!
## 
## 
## Generating affiliation field tag AU_UN from C1:  Done!

1.2 Bibliographic Network Matrices - Journal Names

1.4 a Conceptual Structure Map

Creating a Conceptual Structure Map from the Titles Using the MCA Method with terms mentioned at least 25 times in the title

2 Extracting online transportation safety data

This section provides sources for online open-access transportation data, including both historic and real-time crash-related, traffic flow, and weathaer data. We also provided R codes to read different formats of data and convert them to compatible comma separated value (.csv) files.

2.2 Traffic flow data

2.2.1 Historical data (yearly)

FHWA has provided Annual Average Daily Traffic (AADT) from 2011 to 2017. As an illustration, the following code chunk displays the first five observations of AADT data for Missouri 2017.

The downloaded “shape files” can be converted to different data formats (e.g., .csv) using the following R code.

2.2.2 Real-time data (<= 5 minutes)

There are several sources for getting real-time traffic data. Some of the states in the USA are equipped with loop detectors and video cameras. Departments of Transportation (DoT) can provide this data.

Further, HERE website also provides near real-time traffic data with the limitation of 250,000 APIs per month for free. The HERE Traffic API provides traffic flow and incidents information. It also allows the users to request traffic map tiles. The HERE Traffic API provides four types of traffic data:

  • Traffic Incident Data: the type and location of each traffic incident, status, start and end time, and other relevant data
  • Traffic Map Tile Overlays (Traffic Tiles): pre-rendered map tile overlays with traffic information
  • Traffic Flow Data: real-time traffic flow data, including speed, congestion, geometry of the road segments
  • Traffic Flow Availability: traffic flow information, excluding incidents in an area

The HERE API do not provide a package/interface in R. Queries data from HERE API in R may need messy and highly customized code, so we do not provide R code here.

2.3 Weather data

2.3.1 Historical (daily)

NOAA

2.3.2 The DarkSky API

In this part, we attempt show getting both historical and real-time weather data using DarkSky API. It can be used in both Python and R. Before using the DarkSky API to get weather data, you need to register for a API key on its official website. The first 1000 API requests you make each day are free, but each API request over the 1000 daily limit will cost you $0.0001, which means a million extra API requests will cost you 100 USD.

To get weather data from the DarkSky API, you need to provide 1) latitude, 2) longitude, 3) date and time. Then you can pass these three parameters to the get_forecast_for() function in darksky package in R.

For each observation (a combination of latitude, longitude, date and time), the darksky API returns a list of 3 data.frames:

  1. hourly weather. 24 hourly observations for each 15 weather variables in that day.
  2. daily weathe. 1 observations for each 34 weather variables in that day.
  3. current weather. 1 observations for each 15 weather variables at the assigned time point.

The variables include: apparent (feels-like) temperature, atmospheric pressure, dew point, humidity, liquid precipitation rate, moon phase, nearest storm distance, nearest storm direction, ozone, precipitation type, snowfall, sun rise/set, temperature, text summaries, uv index, wind gust, wind speed, wind direction

hourly weather

daily weather

currently weather

3 Descriptive analytic tools

3.1 Data visualization

3.1.1 Time-series data

Line and stacked graphs

Line graphs are the most commonly used chart for purpose of displaying time-series data, where the x-axis is used to capture time and a transportation-related variable is depicted on the y-axis. The ease-of-use of line charts deteriorates as the number of depicted variables increases. In this case, other time-series based charts are more suitable like ThemeRiver. Here are some examples:

  1. Ferreira et al. (2013) used a line chart to visualize: (a) the number of trips performed by taxi drivers in New York City for 2011 and 2012, and (b) the dollar amount of tips per trip and fare per miles-driven for trips originating in different neighborhoods of the city.
The number of trips performed by taxi in New York City drivers. (Ferreira et al. 2013)

The number of trips performed by taxi in New York City drivers. (Ferreira et al. 2013)

dollar amount of tips per trip and fare per miles-driven (Ferreira et al. 2013)
  1. Guo et al. (2011) added glyphs into ThemeRiver to illustrate time-varying characteristics of a transportation-related variable as well as its directional patterns. The authors used this graph as a traffic pattern recognition tool. They could discovered different traffic volume for two different trajectories with their directions in an intersection by ThemeRiver visualization technique.
Discovering traffic volume patterns by ThemeRiver (Guo et al. 2011)

Radial layout

Radial layout is a visualization technique to display data which exhibits a cyclic behavior. Pu et al. (2013) used this technique to show traffic information in different days and times. In their approach, each ring was used to represent a day, time was shown on the circular axis, and the color was to used to capture low to high traffic volume.

Variation of traffic volume in different days and times (Pu et al. 2013)

Cluster-and-calendar based visualization

Cluster and calendar based visualization approach can be used to depict seasonality in daily patterns. In this approach, the days are clustered based on the hourly data. Then, the average patterns for the clusters are visualized through a line graph with multiple time-series (each corresponding to a cluster and color-coded accordingly). The line chart is supplemented with a calendar heat map, where each day is colored according to the cluster they belong to.

Tsai et al. (2015) divided traffic patterns into eight clusters. These clusters not only identified workday and weekend effects, but it was also able to identify game-day traffic for college football (which are considered major events in the Southeastern U.S. states) and unusual travel on or near major holidays.

Different traffic pattern based on days (Tsai et al. 2015)

Different traffic pattern based on days (Tsai et al. 2015)

The mean of traffic volume for each cluster (Tsai et al. 2015)

Storylines

When data is based on the order and sequence of events instead of using time as the main exploratory variable, Storyline can be a useful technique to visuals them. Tanahashi and Ma (2012) presented a storyline the movie Inception. Encapsulating contours representing the hierarchical worlds introduced in the movie.

A storyline visualization of the movie Inception augmented with spatial information

3.1.2 Spatial data

Symbol maps

In this approach, each symbol on a map represents the position of an object at a given point in time. A popular implementation of a symbol map in transportation safety is within the NHTSA (2018) dashboard for visualizing traffic fatalities. Their dashboard contains a symbol map, and six filters that allow for removing unwanted data from the visualization. The filters can be applied to: person type (occupant vs. non- occupant), month, day, hour, state name, and roadway type.

The location of vehicle occupants killed in speed-related crashes occurring on Saturdays, December, 2016(NHTSA 2018)

Edge bundling

When trajectory data becomes large, visualizing all the data directly on map can be confusing. Edge bundling is a technique which aims to make understandable visualizations in the aforementioned case by grouping similar trajectories together. Cui et al. (2008) used edge bundling technique to display USA immigration information on a map.

U.S. immigration information graph (NHTSA 2018)

Immigration information after utilizing edge bundling approach (NHTSA 2018)

Radial metaphor

When data has periodic pattern, radial visualization techniques (see Draper, Livnat, and Riesenfeld (2009) for more information) are suitable tools. Radial metaphor were used by Zeng et al. (2013) to visualize interchanging traffic patterns among different regions of a city.

Traffic patterns among different regions of a city with different scales (Zeng et al. 2013)

3.1.3 Spatiotemporal data

Space-Time-Cube (STC)

In this method, X and Y axis represents the spatial information and the temporal information is shown by Z axis. Kraak (2003) used STC to show his travel information on an average Thursday in Enschede, the Netherlands.

Travels information (Kraak 2003)

3.1.4 Data with multiple properties

Stacking-based STC

A variant of STC which is capable to display multiple trajectories by stacking them together. Tominski et al. (2012) has shown a traffic jam in San Francisco by Stacking-based STC.

An example of traffic jam in San Francisco (Tominski et al. 2012)

GeoTime

Another variant of STC in which 3-D trajectories represent events, objects and activities. Events, times and locations associated with them are shown with 3-D trajectories (Kapler and Wright 2005).

A 3-D trajectories example by (Kapler and Wright 2005)

3.2 An example of clustering

The following codes attempts to replicate the visual clustering approach by Van Wijk and Van Selow (1999)

Van Wijk, Jarke J., and Edward R. Van Selow. 1999. “Cluster and Calendar Based Visualization of Time Series Data.” In Information Visualization, 1999.(Info Vis’ 99) Proceedings. 1999 IEEE Symposium on, 4-9. IEEE.

A brief example of applying EDA methods on traffic data is provided here. The goal of this example is to illustrate the efficiency of the mentioned tools in the transportation context. There is no predetermined way to utilize these methods. The efficiency of each method highly depends on the nature of the problem. Hence, the challenge is to choose the right tool which fits the best.

3.2.1 Collecting Data

Hourly vehicle counts data is used in this example. It provides the number of vehicles which passed along a particular segment of a road in one hour. Data is extracted from the Georgia Department of Transportation (GDoT) (Georgia Department of Transportation, 2015) for 2015 from station 121-5505 which located in Atlanta. GDoT provides data in separate sheets for each month. After extracting and cleaning data, it was combined to one sheet with 365 rows (days) and 24 columns (hours). Data can be downloaded from GDoT.

3.2.2 Clustering

It is almost impossible to understand raw data and also discover interesting patterns in it by just looking at 8760 (370 * 24) data cells. Hence, K-means clustering method is utilized here to present data in a more understandable format. K-means clustering is a common technique to explore data and discover patterns by grouping similar data to predefined (k) number of clusters. K-means clustering aims to group data into k clusters in a way to minimize the within-cluster sum of squares (WCSS). To find the optimal number of clusters, we have used a method that was suggested by Pham, Dimov, and Nguyen (2005). According to the following graph two is the best number of clusters to group this data.

## 
##   1   2 
## 119 246

3.2.3 Visualization

Now, k-means clustering can be applied. The output of this step is a column which its value is either one or two,indicating that each row of data (day) belongs to cluster one or two. Now data is divided into two groups. However, still we need to transfer data to a visual format to somehow validate and guide the clustering process. Since our data contains temporal information, we have used Cluster Calendar View visualization technique which is introduced by Van Wijk and Van Selow (1999). In this technique, a calendar represents the temporal information of data and by using color coding, differences between clusters are distinguished. The following graph shows a cluster calendar view for our data. It clearly has found meaningful patterns in the vehicle counts data. Weekends and weekdays have different traffic patterns. Besides, it has captured some of the holidays. For example, the 4th of July (Independence Day) which is a weekday, is colored by light blue. It means that this day has a similar traffic pattern with weekends. In addition, the clustering method has identified other holidays like Martin Luther King Day, Memorial Day, Labor Day, Thanksgiving Day and Christmas Day.

Furthermore, a line chart (following graph) is used to show the average hourly traffic data for the two clusters. Results show that each cluster has different peaks and valleys. On the weekdays, 7 AM and 4 PM have the greatest number of vehicles which can be explained by the official working hours. On the other hand, on weekends, the traffic peak is around 1 PM which maybe refers to some people going out for lunch.

To sum up, it seems that K-means clustering method was very efficient here. We applied raw data as inputs to this method and as outputs we could discover patterns (weekdays and weekends traffic patterns) and also with the help of visualization technique we obtained a considerable information about the data.

4 Statistical models

Logistic regression and Poisson regression are two most commonly used statistical models in traffic safety studies. In this section, we provides introductory statistical theories on the two models, and

4.1 Logistic regression

The most common model for traffic safety studies is logistic regression, where the response is \(Y_i=1\) if a crash occurred in a given segment/time period, and \(Y_i=0\) if no crash occurred. In logistic regression we assume that the logit of the probability of a traffic crash is a combination of a linear predictor variables:

\[ Y_i \sim \text{Bernoulli}(p_i)\\ \text{logit}(p_i) = \log \left( \frac{p_i}{1-p_i}\right) = \textbf{X}^{\prime} \boldsymbol{\beta} \]

Here

\[ \textbf{X}^{\prime} \boldsymbol{\beta} = \beta_0 + \sum_{j=1}^{p} x_{ij} \beta_j \]

where the \(x_{ij}\) are covariates. A simple linear regression can be conducted using the following R code.

For a detailed explanation on interpreting the results of a logistic regression, the readers can refer to the example by the UCLA.

4.2 Poisson regression

Another widely used model for traffic safety studies is Poisson regression, where the response \(Y_i\) is the number of crashes.

\[ Y_i \sim \text{Poisson}(T*\lambda)\\ \log\lambda = \textbf{X}^{\prime} \boldsymbol{\beta} \]

Here \(T\) is the length/time of a trip. It is an exposure variable that indicates the number of crashes could have occurred.

## 
## Call:
## glm(formula = Y ~ fatigue + traffic + precipitation, family = "poisson", 
##     data = pois_dat, offset = trip)
## 
## Deviance Residuals: 
##     Min       1Q   Median       3Q      Max  
## -2.9729  -0.7845  -0.4613   0.8670   3.0791  
## 
## Coefficients:
##               Estimate Std. Error z value Pr(>|z|)    
## (Intercept)   -9.60355    0.23534 -40.808   <2e-16 ***
## fatigue        0.45258    0.05234   8.647   <2e-16 ***
## traffic        0.21955    0.02016  10.890   <2e-16 ***
## precipitation  0.33890    0.33963   0.998    0.318    
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## (Dispersion parameter for poisson family taken to be 1)
## 
##     Null deviance: 856.41  on 499  degrees of freedom
## Residual deviance: 673.09  on 496  degrees of freedom
## AIC: 1167
## 
## Number of Fisher Scoring iterations: 6

For a detailed explanation on interpreting the results of a Poisson regression, the readers can refer to the example by the UCLA.

References

Cui, Weiwei, Hong Zhou, Huamin Qu, Pak Chung Wong, and Xiaoming Li. 2008. “Geometry-Based Edge Clustering for Graph Visualization.” IEEE Transactions on Visualization and Computer Graphics 14 (6): 1277–84.

Draper, Geoffrey M, Yarden Livnat, and Richard F Riesenfeld. 2009. “A Survey of Radial Methods for Information Visualization.” IEEE Transactions on Visualization and Computer Graphics 15 (5): 759–76.

Ferreira, Nivan, Jorge Poco, Huy T Vo, Juliana Freire, and Cláudio T Silva. 2013. “Visual Exploration of Big Spatio-Temporal Urban Data: A Study of New York City Taxi Trips.” IEEE Transactions on Visualization and Computer Graphics 19 (12): 2149–58.

Guo, Hanqi, Zuchao Wang, Bowen Yu, Huijing Zhao, and Xiaoru Yuan. 2011. “Tripvista: Triple Perspective Visual Trajectory Analytics and Its Application on Microscopic Traffic Data at a Road Intersection.” In 2011 Ieee Pacific Visualization Symposium, 163–70. IEEE.

Kapler, Thomas, and William Wright. 2005. “Geotime Information Visualization.” Information Visualization 4 (2): 136–46.

Kraak, Menno-Jan. 2003. “The Space-Time Cube Revisited from a Geovisualization Perspective.” In Proc. 21st International Cartographic Conference, 1988–96. Citeseer.

NHTSA. 2018. “FARS Speeding Data Visualization.” United States Department of Transportation, https://www.nhtsa.gov/press-releases/usdot-releases-2016-fatal-traffic-crash-data.

Pham, Duc Truong, Stefan S Dimov, and Chi D Nguyen. 2005. “Selection of K in K-Means Clustering.” Proceedings of the Institution of Mechanical Engineers, Part C: Journal of Mechanical Engineering Science 219 (1): 103–19.

Pu, Jiansu, Siyuan Liu, Ye Ding, Huamin Qu, and Lionel Ni. 2013. “T-Watcher: A New Visual Analytic System for Effective Traffic Surveillance.” In 2013 Ieee 14th International Conference on Mobile Data Management, 1:127–36. IEEE.

Tanahashi, Yuzuru, and Kwan-Liu Ma. 2012. “Design Considerations for Optimizing Storyline Visualizations.” IEEE Transactions on Visualization and Computer Graphics 18 (12): 2679–88.

Tominski, Christian, Heidrun Schumann, Gennady Andrienko, and Natalia Andrienko. 2012. “Stacking-Based Visualization of Trajectory Attribute Data.” IEEE Transactions on Visualization and Computer Graphics 18 (12): 2565–74.

Tsai, Yao-Te, Huw D Smith, Stephen M Swartz, and Fadel M Megahed. 2015. “Using Visual Data Mining in Highway Traffic Safety Analysis and Decision Making.” Journal of Transportation Management 26 (1): 5.

Van Wijk, Jarke J, and Edward R Van Selow. 1999. “Cluster and Calendar Based Visualization of Time Series Data.” In Proceedings 1999 Ieee Symposium on Information Visualization (Infovis’ 99), 4–9. IEEE.

Zeng, Wei, Chi-Wing Fu, Stefan Müller Arisona, and Huamin Qu. 2013. “Visualizing Interchange Patterns in Massive Movement Data.” In Computer Graphics Forum, 32:271–80. 3pt3. Wiley Online Library.

  1. Department of Epidemiology and Biostatistics, Saint Louis University. Email address miao.cai@slu.edu↩

  2. Department of Industrial and Systems Engineering, Auburn University. Email address azm0127@auburn.edu↩

  3. Carey Business School, Johns Hopkins Universitymza0052@auburn.edu↩

  4. Farmer School of Business, Miami University. Email address fmegahed@miamioh.edu.↩

Miao Cai, M.S.1

Amir Mehdizadeh, M.S.2

Mohammad Ali Alamdar Yazdi, Ph.D.3

Fadel M. Megahed, Ph.D.4

2019-02-20